Statistical Mechanics of Adaptive Weight Perturbation Learning

The weight perturbation learning was proposed as a learning rule which adds perturbation to the variable parameters of learning machines. Generalization performance of the weight perturbation learning was analyzed by statistical mechanical methods. The weight perturbation learning has the same asymptotic generalization property as the Perceptron learning. In this paper we consider difference between the Perceptron learning and the AdaTron learning which are well-known learning rules. Applying the consideration to the weight perturbation learning, we propose the adaptive weight perturbation learning. The generalization performance of the proposed rule is analyzed by statistical mechanical methods. Consequently, it is shown that the proposed learning rule has an outstanding asymptotic property corresponding to the AdaTron learning.